Automatically Identifying Usability Problems in E-commerce Websites

Managers at e-commerce firms are in need of proven methods for evaluating the usability of their websites. So, one of the most pressing issues is whether the design of their online storefronts is effective, and if not, which areas require attention and improvements. However, current usability evaluation methods (e.g., user testing, inspection, inquiry) are not well suited to the task at hand. This paper proposes a new approach to website evaluation, which is grounded in the economic theory of production. We conceptualize human-computer interaction during online shopping as an economic production process in which customers are making use of various functionalities provided by the e-commerce website in order to complete a purchase transaction. This view enables us to formulate a novel perspective on website usability — the ability to transform inputs (i.e., use of website functionalities) into outputs (i.e., completed purchase transactions). We use data envelopment analysis (DEA) as the methodological vehicle for evaluating e-commerce websites and propose two new metrics, Inefficiency Breadth and Unit Inefficiency to help automatically identify website functionalities that are potentially problematic. The value of the proposed method is illustrated by applying it to the evaluation of a real-world ecommerce website.